Two-Level Dirichlet Posterior + Per-Order OBCL + Phrase Cache

val_bpb: 0.11556 (3-seed mean, std 0.0000057) | ~15.1 MB | 8xH100 SXM

3-seed validation

Techniques

• Two-level Dirichlet-Multinomial posterior mixing (neural → n-gram → phrase)
• Per-order OBCL concentrations: [50.0, 50.0, 6.95, 2.98, 2.05, 2.05, 2.05, 1.86, 1.86, 1.86, 1.86, 1.86, 1.86, 1.86]
• Phrase suffix matching at probe lengths [20, 16] with Dirichlet concentration 1.0
• 15-gram backoff (orders 2-15, 4M hash buckets)
• Complementary training (alpha=0.50, orders 2-5)
• EBLS architecture (3 shared x 3 loops + 2 unique = 11L)
• GPTQ int6 + LZMA compression
• EMA 0.997 + SWA weight averaging

Compliance

• Training: 560s on 8xH100 (within 600s)
• Eval: 419s worst case (within 600s)
• Artifact: 15,077,877 bytes (within 16,000,000)
• All caches strictly backward-looking (causal)
• Score-first evaluation
• No training data accessed during evaluation

Credits

Built on the community's work:

• @signalrush (PR Record: 11L EMA + GPTQ-lite + warmdown3500 + QAT@0.15 (val_bpb=1.1233) #414) — GPTQ + EMA + warmdown
• @Robby955 (PR Record: Two-Level Dirichlet Posterior Mixing with Per-Order OBCL -- 0.1156 BPB #900) — Dirichlet smoothing, OBCL, phrase cache
• @himanshudongre (PR Record: Two-Pass N-gram Rescoring (val_bpb 0.1434) #846) — two-pass rescoring
• @deanbrr (PR Record: 5-gram Eval Cache + LeakyReLU² + Parallel Muon val_bpb: 1.0920 (3-seed mean, std 0.0007) | ~15.9 MB | 8×H100 SXM #659) — original N-gram cache concept
• @newjordan (PR Podracing: 1.0461 BPB (3-seed mean) #674) — first legal implementation
• @pentxayc (PR Record: 0.4416 BPB -- Complementary Training + Backoff N-gram Mixer #803) — complementary training